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California's College of the Canyons has used the four 
Matnematics Diagnostic Testing Project (MDTP) tests to assess 
students' abilities in basic and college math since spring 1989. 
These four reports dated June 1993, July 1993, May 1994, and June 
1994 examine the predictive validity of the MDTP tests and the cut 
scores used to place students. The first report compares outcomes for 
students in fall 1991 and fall 1992 who followed placement advice to 
those who did not, finding that for MDTP tests 1, 2, and 4, a higher 
percentage of students who followed placement advice were likely to 
receive satisfactory grades than those not following advice. The 
second and third reports examine the predictive validity of the MDTP 
tests in studies conducted July 1993 and May 1994, with evidence from 
the 1994 analysis indicating that the MDTP tests exceeded the minimal 
standards for the predictive validity of placement instruments. This 
report also recommends that MDTP versions 1, 2, and 4 be used to 
direct students into mathematics classes. The fourth report examines 
the best possible cut scores for the placement tests by using scatter 
plots to separate successful versus unsuccessful course outcomes, 
finding the current scores reasonable, and recommending that 
sub-scores not be used in placement but that their results be given 
to students and faculty as feedback. (KP) 
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EXPERIENCE TABLES FOR THE 
MATHEMATICS DIAG STIC TESTING PROJECT 
PLAC ENT TESTS 

Design 19 



A practical evaluation of the usefulness of the mathematics 
placement tests is shown by comparing the outcomes of 
students who followed the placement advice to students who 
did not. This is a non-technical way to represent 
predictive validity data for use in counseling sessions with 
students. 

SUBJECTS 

The research was limited to first-time college students who 
had taken one of the four math placement tests and enrolled 
in a target math course during that semester, with no 
intervening coursework. The data for these computations 
represent Fall 1991 and Fall 1992 enrollments. 

While it is recommended that a sample be composed of a 
minimum of 200 students, the samples for each of the four 
classes considered were much smaller, ranging from 27 to 78 
students. Larger samples are preferable and will yield more 
trustworthy results. 

PROCEDURE 

End-of -course grades were used as the criterion measure. 
Grades were classified as either successful/satisfactory (C 
or higher) or unsuccessf ul/unsatisf actory (D, F or W) . 

Assessment scores were classified in two different ways. 
First, we classified students by whether they had achieved a 
score on the placement exam which made them fully prepared 
for the math course. In the case of the MDTP1 - Algebra 
Readiness Test, the math faculty established a minimum score 
of 35 as the score needed by a student to be fully prepared 
for Math 060 - Elementary Algebra. These students were 
classified as "Advice Followers." Students who scored below 
35, but chose to enroll in the course anyway, were 
classified as "Non-Advice Followers." 
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Percentage of Satisfactory Grades 
Fall 1991 and Fall 1992 Data 




Course/Placement Test 



□Advice Followers ■Non-Advice Followers 



Advice Followers: 



MDTP1 score of 35 or higher 
MDTP2 score of 35 or higher 
MDTP3 score of 33 or higher 
MDTP4 score of 28 or higher 



Satisfactory Grades: 



A, B or C 
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EXPERIENCE TABLES 

Assessment scores were then set up in intervals , and the 
percentage of students receiving successful course grades (C 
or higher) shown for each intervals Because the tables 
summarize previous experience about student success, they 
are called experience tables. 

We used the cut scores established by the mathematics 
faculty as the intervals for each of the math placement 
exams • 



Experience Table 1 
MDTP1 - Algebra Readiness Test 

Percentage Successful in 
Assessment Score M ath 060 - Elementary Algebra 

0-25 18% 

26 - 29 (Needs much review) 22% 

30 - 34 (Needs some review) 47% 

35 - 50 (Fully prepared) 40% 

Total Group 35% 
N=78 



Experience Table 2 
MDTP2 - Elementary Algebra Diagnostic Test 

Percentage Successful in 
Ass e s smmt__Score M ath 070 - Int ermediate ..Algebra 

0-25 38% 

26 - 29 (Needs much review) 47% 

30 - 34 (Needs some review) 35% 

35 - 50 (Fully prepared) 60% 

Total Group 47% 
N=72 
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Experience Table 3 
MDTP3 - Intermediate Algebra Diagnostic Test 

Percentage Successful in 
Assessment Score Math 102 - Trigonometry 

0-23 63% 

24 - 27 (Needs much review) 44% 

28 - 32 (Needs some review) 67% 

33 - 45 (Fully prepared) 40% 

Total Group 54% 
N=41 



Experience Table 4 
MDTP4 - Pre-Calculus Diagnostic Test 

Percentage Successful in 
Assessment Score Math 211 - Calculus I 

0-23 0% 

24 - 26 (Needs much review) 50% 

27 - 27 (Needs some review) 60% 

28 - 40 (Fully prepared) 92% 

Total Group 67% 
N=27 



DISCUSSION 

BAR GRAPH 

For all but the MDTP3 , the bar graph illustrates that a 
higher percentage of students who followed the advice of the 
placement test were likely to receive satisfactory grades on 
the target course. For example, 60% of the MDTP2 Advice 
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Followers received satisfactory course grades in Math 070 - 
Intermediate Algebra, whereas only 40% of the Non-Advice 
Followers received grades of C or higher. 

Contrary to this, a higher percentage of the MDTP3 
Non-Advice Followers received satisfactory grades in Math 
102 than did Advice Followers (56% versus 40%). It i£ 
difficult to know why there is a reversal of the expected 
outcomes. It may simply be that too little data is 
available, and the low cell counts are yielding percentages 
influenced by chance. Caution needs to be used in 
interpreting these data because of the small number of 
students in the sample (41). 



EXPERIENCE TABLES 

When we looked at the success rate of students by ^ assessment 
score intervals, generally, students who scored higher on a 
placement test were more likely to receive satisfactory 
course grades. Experience Table 2 shows that while only 38% 
of the students who scored from 0 to 25 received grades of C 
or higher, this compared to 60% of the students who were 
considered fully prepared (35-50). 

This was not true for all placement tests, however. 
Experience Table 3 shows that a higher percentage of 
students who scored in the 28 to 3 2 interval on the MDTP3 - 
Intermediate Algebra Diagnostic Test, were more likely to 
receive a satisfactory course grade than were the students 
who were thought to be fully prepared (67% versus 40%). 
Caution must be used in interpreting these data because of 
the small number of students in the sample (41). 



RECOMMENDATION 

These data are based on small samples and therefore may 
prove to be untrustworthy. This caution is emphasized for 
Experience Tables 3 and 4 with samples of only 41 and 27 
students , respectively . 

Math 070 - Intermediate Algebra, serves as a prerequisite 
for Math 102 - Trigonometry and Math 103 - College Algebra. 
Many students move into those courses because of their 
successful completion of the prerequisite, not because of 
their score on the placement test. This will be true of any 
higher level math course. For this reason, it may take time 
to assemble a database with 200 or more students for the 
MDTP3 and MDTP4. 
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Since the college has only recently started collecting 
social security numbers for its students, it has proven 
difficult to retrieve and match assessment data with course 
data prior to Fall 1991. We recommend that the math 
research data bases be expanded by adding placement and 
course data for Spring 1992, Spring 1993 and Fall 1993 , to 
further increase the size of the samples. This would 
provide the college with comprehensive research data bases 
for each of the four math placement tests from Fall 1991 
forward. The expanded databases would assist the college in 
making more trustworthy judgments about the ability of the 
math placement tests to predict student success in the 
target courses. 
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PREDICTIVE VALIDITY STUDIES 
OF THE POUR 

MATHEMATICS DIAGNOSTIC TESTING PROJECT ASSESSMENT TESTS 

July 1993 



This report summarizes the predictive validity studies 
carried out during July of 1993. The four tests of the 
Mathematics Diagnostic Testing Project (MDTP) were examined 
to determine their ability to predict student success in 
both basic skills and college level mathematics courses 
offered at College of the Canyons. 

Matriculation regulations require that community 
colleges which adopt a standardized test for making 
placement recommendations must qualify its use locally. The 
test must be a valid predictor of future course success 
given the unique features of the college curriculum, its 
instructors, and student population. Establishing a clear 
relationship between the test score and subsequent student 
performance in a particular course for which placement 
recommendations were made is predictive validity. A 
correlation of no less than +.35 is needed to conclude that 
the test instrument has sufficient predictive validity for 
advisement purposes. 

One measure of course success was used, the final 
course grade. In all cases, Ws were recoded as failure and 
kept in the computation. The resulting correlations are 
shown in the tables below. The college has used the four 
tests of the MDTP series since Spring 1989 and the data for 
these computations represent information from first-time 
students from four semesters (Fall 1991, Spring 1992, Fall 
1992 and Spring 1993). Since placement test scores were in 
use, the correlation values were adjusted by using a 
correction for restriction of range. 





MDTP1 - 


Algebra Readiness Test 




TJST 


COURSE 


MEAN 


r- SCORE TO 
GRADE 


NUMBER OF 
STUDENTS 


MDTP1 


Math 025 


18.0 


.47 


117 


MDTP1 


Math 057 


26.5 


.47 


47 


MDTP1 


Math 060 


32.0 


.42 


98 
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MDTP2 - Elementary Algebra Test 



TEST 

MDTP2 
MDTP2 
MDTP2 
MDTP2 



COURSE 

Math 025 
Math 060 
Math 063 
Math 070 



MEAN r- SCORE TO 
GRADE 

10.0 .64 

19,9 .44 

No data available 

30.6 .22 



NUMBER OF 
STUD ENTS 

13 

150 



MDTP3 - Intermediate Algebra Test 

IE.s_T C OURS E MEAN r- SCORE TO NUMBER OF 

GRADE STUDENT S 



MDTP3 


Math 


060 


12. 


5 


-.02 


68 


MDTP3 


Math 


063 


NO 


data 


available 




MDTP3 


Math 


070 


18, 


.5 


.33 


120 


MDTP3 


Math 


102 


26, 


.9 


.11 


50 


MDTP3 


Math 


103 


27 


.3 


.20 


29 


MDTP3 


Math 


140 


24 


.3 


-.51 


16 


MDTP3 


Math 


240 


31 


.0 


-.95 


3 
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TEST 



MDTP4 
COURSE 



Pre-Caiculus Test 
MEAN r-„gSgpRE TO 

Shade 



NUMBER OF 
STUDENTS 



MDTP4 


Math 


060 


10.0 


1.00 


3 


MDTP4 


Math 


063 


No data 


available 




MDTP4 


Math 


070 


14.8 


.05 


5 


MDTP4 


Math 


102 


21.0 


.37 


19 


MDTP4 


Math 


103 


20.9 


.82 


9 


MDTP4 


Math 


140 


25.0 


.97 


5 


MDTP4 


Math 


211 


27.9 


.79 


29 


MDTP4 


Math 


240 


19.0 


.00 


2 



DISCUSSION 

MDTP1 - Algebra Readiness Test 

These findings offer initial support that the MDTP 
Algebra Readiness Test has predictive validity for Math 025 , 
Math 057 and Math 060. 

MDTP 2 - Elementary Algebra Test 

These findings offer initial support that the MDTP 
Elementary Algebra Test has predictive validity for Math 
060. 

The small number of students who enrolled in Math 025 
after taking this placement test makes it impossible to 
judge the validity of the test for this course placement. 
The MDTP Elementary Algebra Test has not yet shown to have 
predictive validity for Math 070 (r=.22). 

MDTP 3 - Intermediate Algebra Test 

The MDTP Intermediate Algebra Test has not yet shown to 
have predictive validity for any of the following 
mathematics courses: Math 060 , Math 070 , Math 102 , Math 
103 , Math 140 or Math 240. 
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MDTP4 - Pre-Calculus Tes t 

These findings offer initial support that the MDTP 
Pre-Calculus Test has predictive validity for Math 211. 

The small number of students in the samples makes it 
impossible to determine with any accuracy the predictive 
validity of the Pre-Calculus Test for the following 
mathematics courses: Math 060, Math 070, Math 102, Math 
103, Math 140 and Math 240. 

The reader is reminded that these correlations apply 
only to the ability of the MDTP tests to predict the final 
grade performance of first-time college students. Further 
predictive validity computations will be performed as 
additional data become available. Where predictive validity 
has not been shown, it may be necessary to carry out content 
validity studies with the mathematics faculty. 



CONCLUSIONS 

MDTP1 - Algebra Readiness Test 

The MDTP Algebra Readiness Test appears to be an 
appropriate test for Math 025, Math 057 and Math 060. 

MDTP2 - Elementary Algebra Test 

The MDTP Elementary Algebra Test appears to be an 
appropriate test for Math 060. 

MDTP 3 - Intermediate Algebra Test 

The MDTP Intermediate Algebra Test may not be an 
appropriate test for any of the mathematics courses studied, 
although further data collection and analysis may qualify 
it. 

MDTP4 - Pre-Calculus Test 

The MDTP Pre-Calculus Test appears to be an appropriate 
test for Math 211. 

The college should continue to evaluate the MDTP tests 
and/or multiple measures which may provide the most accurate 
advice possible when directing students into mathematics 
classes. 
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PREDICTIVE VALIDITY STUDIES 
OF THE 

MATHEMATICS DIAGNOSTIC TESTING PROJECT ASSESSMENT TESTS 

May 1994 

This report summarizes the predictive validity studies 
carried out during May of 1994. The four tests of the 
Mathematics Diagnostic Testing Project (MDTP) were examined 
to determine their ability to predict student success in 
both basic skills and college level mathematics courses 
offered at College of the Canyons. 

The college has used the four tests of the MDTP series 
since Spring 1989 and the data for these computations 
represent information from first-time students from five 
semesters (Fall 1991, Spring 1992, Fall 1992, Spring 1993 
and Fall 1993) . 

Matriculation regulations require that community 
colleges which adopt a standardized test for making 
placement recommendations must qualify its use locally. The 
test must be a valid predictor of future course success 
given the unique features of the college curriculum, its 
instructors, and student population. Establishing a clear 
relationship between the test score and subsequent student 
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performance in a particular course for which placement 
recommendations were made is predictive validity. A 
correlation of no less than +.35 is needed to conclude that 
the test instrument has sufficient predictive validity for 
advisement purposes. 

Descriptive Statistics . The data collected for this study 
are summarized in Table 1. These data constitute an 
important step toward the development of local norms for the 
MDTP test battery at College of the Canyons. The mean score 
and skewness provide a clue about the relative degree of 
difficulty of the tests. Positively skewed score 
distributions (+), or those where the cases are clustered 
more to the left of the mean, indicate that sampled students 
tend to find the test difficult. Thus, the MDTP 3 test was 
clearly more difficult for students in Math 102 than was the 
MDTP4 test for students in Math 211. In relative terms the 
least difficult test was the MDTP1 Algebra Readiness Test, 
for students in Math 057. 
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Table 1. 

Descriptive Statistics for MDTP Placement Tests, 

By Course Level 



TEST COURSE 



H 



MEAN 



MEDIAN 



STANDARD 
DEVIATION 



SKEWNESS 



MDTP1 


Math 


025* 


165 


18 


.44 


19 


.00 


5 


.82 


-0 


. 30 




Math 


057 


60 


26 


. 62 


27 


.00 


4 


. 96 


-0 


.67 




Math 


060* 


135 


32 


.68 


33 


.00 


6 


.82 


-0 


.43 


MDTP2 


Math 


060 


226 


20 


.69 


20 


.00 


6 


.10 


0 


. 38 




Math 


070* 


115 


31 


. 10 


32 


.00 


7 


. 28 


-0 


.53 


MDTP3 


Math 


060 


82 


12 


.44 


12 


.00 


3 


.62 


0 


. 25 




Math 


070 


172 


18 


.74 


19 


.00 


5 


.00 


-0 


. 39 




Math 


102* 


70 


27 


.09 


27 


.00 


4 


. 96 


0 


. 38 




Math 


103* 


38 


27 


. 76 


29 


.00 


6 


. 77 


-0 


. 50 




Math 


140* 


30 


23 


. 90 


23 


.00 


5 


.02 


-0 


. 11 




Math 


240* 


5 


31 


.40 


32 


.00 


2 


.41 


-0 


. 29 


MDTP4 


Math 


102 


23 


20 


.61 


21 


.00 


5 


.78 


-0 


.52 




Math 


103 


16 


19 


.88 


18 


.50 


6 


.99 


0 


.43 




Math 


140 


11 


20 


. 55 


18 


.00 


5 


.80 


0 


. 26 




Math 


211* 


40 


28 


.25 


28 


.00 


4 


.74 


-0 


.09 




Math 


240 


4 


20 


. 25 


20 


.00 


1 


.50 


0 


.14 



* Primary target course(s) for placement test 
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Predictive Validity * The primary index of the strength of 
the relationship between test scores (predictor) and course 
grades (criterion) is the predictive validity coefficient, 
or the correlation between predictor and criterion. 

Two correlation coefficients indicating predictive 
validity were computed for each course — one between the 
test scores and course grades with all withdrawals (Ws) 
converted to Fail (F) , and the second between the test 
scores and course grades with withdrawals deleted from the 
analysis. The resulting correlations are shown in the 
table on the following page. Since placement test scores 
were in use, the correlation values were adjusted by using a 
correction for restriction of range. 
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Table 2 

Relationship Between MDTP Test Scores and Grades, 

By Course 



TEST 



MDTPl - Algebra Readiness Test 

COURSE r- SCORE TO GRADE 

W=FAILURE Ws EXCLUDED 



MDTPl 



Math 025* 
Math 057 
Math 060* 



.44 (124) 
.47 ( 41) 
.42 (134) 



.43 (91) 
.58 (29) 
.42 (86) 



MDTP2 - Elementary Algebra Test 



MDTP2 


Math 060 
Math 070* 


.35 (226) 
.24 (114) 


. 33 
.13 


(179) 
( 73) 






MDTP 3 - Intermediate Algebra 


Test 






MDTP 3 


Math 060 


.18 ( 81) 


. 35 


( 62) 






Math 070 


.40 (171) 


.35 


(127) 






Math 102* 


.15 ( 70) 


-.04 


( 49) 






Math 103* 


.26 ( 38) 


. 18 


( 31) 






Math 140* 


-.27 ( 29) 


-.04 


( 24) 






Math 240* 


-.79 ( 5) 


-.79 


( 5) 





MDTP4 - Pre-Calculus Test 



MDTP4 Math 102 

Math 103 
Math 140 
Math 211* 
Math 240 



.51 (23) 

.54 (16) 

.83 (11) 

.75 (40) 

-.99 ( 4) 



.50 (22) 
.48 (15) 
.82 ( 8) 
.68 (30) 
-.99 ( 4) 



* Primary target course(s) for placement test 
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DISCUSSION 

As can be seen, there are only minor differences 
between the validity coefficients computed for each course 
with the two approaches. With the exception of Math 240 - 
Math Analysis for Business and Social Science, that had only 

five students in the MDTP3 and four students in the MDTP4 

If 

correlations, t^median predictive validity value across 
all levels is .41 , ranging from a low of -.27 for MDTP3 - 
Math 140 (with Ws included) to a high of .83 for MDTP4 - 
Math 140 (with Ws included). Minimal standards are for 
predictive validity coefficients for placement instruments 
to be at least .35 (California Community Colleges, 1991). 
Evidence from the current analysis indicates that the MDTP 
tests meet and even exceed this standard at College of the 
Canyons. This is a critical step in the local validation of 
assessment instruments in that without at least a moderate 
overall relationship between test scores and course grades, 
a test would be no better at generating placement 
recommendations than would a random method of doing so. 
Indeed, given the attenuation of the validity coefficient 
due to less than perfect reliability of both predictor and 
criterion, and the restriction of range due to the use of 
the test scores for placement, the obtained predictive 
validity coefficients are particularly impressive. 



MATRICULATION 



ERIC 



The reader is reminded that these correlations apply 
only to the ability of the MDTP tests to predict the final 
grade performance of first-time college students. 

CONCLUSIONS 

The college should continue to evaluate the validity of 
the MDTP tests and determine multiple measures which may 
provide the most accurate advice possible when directing 
students into mathematics classes. 



References : 
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West College , February 1992. 

Jerry Rudmann, A Study of Math 251 and Math 253 Assessment 
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College, May 1991. 
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VALIDATING PLACEMENT RULES 
FOR THE MDTP TESTS 

June 1994 

The report entitled Predictive Validity Studies of the 
Mathematics Diagnostic Testing Project Assessment Tests (May 
1994) showed the adequate overall relationship between test 
scores and course grades, supporting the use of MDTP test 
scores as predictors of student performance at College of 
the Canyons. 

The next issue is a very practical one — how best to 
use the tests for making placement recommendations. This 
involves identifying cut scores. At what point in the 
continuum of test scores do we recommend placement of 
students in a given course? For example, are students who 
receive scores of 35 to 50 on the Algebra Readiness Test 
fully prepared to undertake Math 060 - Elementary Algebra? 
Should the score range be broader or narrower? No method of 
identifying cut scores is flawless. With typical validities 
of .35, only about 12 percent of the variance in student 
performance is associated with the test scores. A great 
many factors not tapped by the tests contribute to course 
grades. The goal of the process is to improve prediction of 



success. 

METHOD 

The primary method used for establishing cut scores 
involves comparing success rates for contrasting groups over 
a range of scores. More specifically, the contrasting 
groups are defined as "successful students" or those who 
earned grades of A, B, C and CR, whereas persons with grades 
of D, F, NC or W (withdrawal) are categorized in the 
"unsuccessful" group. Given this distinction, it is then 
possible to determine the proportion of students with 
specific scores, and hence for score ranges, that were 
successful. Cut scores are then evaluated in terms of their 
ability to "predict" success (i.e. maximize the difference 
between success rates for persons scoring above a value and 
those scoring below it) . 

Subjects . The research participants were 1,421 students who 
were tested and enrolled in one of several target 
mathematics courses during that subsequent semester; 568 
took the Algebra Readiness Test, 356 took the Elementary 
Algebra Test, 391 took the Intermediate Algebra Teat and 106 
took the Pre calculus Test. The four MDTP databases each 
contain five semesters .of data (Fall 1991, Spring 1992, Fall 
1992, Spring 1993 and Fall 1993). The students were 
first-time college students. Participating students were 
placed in mathematics courses using existing placement 
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procedures, including the use of placement test scores, 
End-of-course grades were used as the student performance 
criterion. 

RESULTS 

Placement Rule s, The scatter diagrams presented in Figures 
1 through 7 show the relationship between test scores and 
success in each target class. These provide an initial 
basis for establishing trial cut scores to minimize 
placement errors. On the scatter plots, a number represents 
one or more coincident data points while letters designate 
two-digit numbers, i.e. A=10, B=ll, C=12 etc. Withdrawals 
were treated as unsuccessful course outcomes in this 
analysis of dichotomized grades. 
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Figure 1. 

Math, 060 Success by MDTP1 - Algebra Readiness Test 



College of the Canyons 

SCATTER DIAGRAM 

Dala: Math 01 F91 - F93 

Course: HATH 868 

Test: MATH 01: ALGEBRA READINESS 
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Figure 2. 

Math 060 Success by MDTP2 - Elementary Algebra Test 
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Figure 3. 

Math 070 Success by MDTP2 - Elementary Algebra Test 
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Figure 4. 

Math 070 Success by MDTP3 - Intermediate Algebra Test 
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Figure 5. 

Math 102 Success by MDTP3 - Intermediate Algebra Test 
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Figure 6. 

Math 102 Success by MDTP4 - Pre-Calculus Test 
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Figure 7. 

Math 211 Success by MDTP4 - Pre-Calculus Test 
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The scatterplots were partitioned using a variety of 
possible cut scores. The partitions produced placement 
classification tables for each trial cut score. The table 
partitions the data from the scatterplot into four 
quadrants. The upper right and lower left quadrants 
represent correct placements. The other cwo quadrants 
represent placement errors. 

The analysis showed that there were several cut scores 
for each course that were optimum in placement accuracy. 
The recommended cut scores shown in Table 1 were set in the 
lower part, of the optimum range for each course. For 
students who score from 0 to 9 on the MDTP3 and MDTP4, we 
recommend that they take the lower level math test. The low 
score represents performance at the level of chance. The 
student will have a better chance of receiving an accurate 
math course placement recommendation from the lower level 
test. 

Table 2 provides the currently used cut scores, without 
the use of sub-scores. 
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Table 1 . 

Recommended MDTP Placement Test Cut Scores by Course 



Test 


Score 


Range 


Recommended 
Placement 


% of Study Sample 
in Score Range N 


MDTP1 


0 




29 


Math 


025 


53.2% 


302 




30 




50 


Math 


060 


46.8% 


266 


MDTP2 


0 


- 


17 


Math 


025 


29.2% 


104 




18 




-J J 


Math 


060 


57.0% 


203 




34 


mm 


50 


Math 


070 


13.8% 


49 


MDTP3 


u 




9 






D . O * 


ZD 




10 




20 


Math 


060 


45.3% 


177 




21 




28 


Math 


070 


34.5% 


135 




29 




45 


Math 


102 


13.6% 


. 53 


MDTP4 


0 




9 


Take 


MDTP 3 


4.7% 


5 




10 




18 


Math 


070 


22.7% 


24 




19 




24 


Math 


102 


29.2% 


31 




25 




40 


Math 


211 


43.4% 


46 
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Table 2. 

Current MDTP Test Cut Scores by Course 



Test 


Score 


Range 


Recommended 
Placement 


% of Study Sample 
in Score Range 


N 


MDTP1 


0 




25 


Math 


025 


37.5% 


213 




26 




50 


Math 


060 


62.5% 


355 


MDTP2 


0 


- 


13 


Math 


025 


12.1% 


43 




14 


- 


25 


Math 


060 


49.7% 


177 




26 




50 


Math 


070 


38.2% 


136 


MDTP3 


0 


- 


16 


Math 


060 


31.2% 


122 




17 


- 


19 


Math 


070 


16.1% 


63 




20 




23 


Math 


140 


19.5% 


76 




24 




45 


Math 


102 


33.2% 


130 


MDTP4 


0 




12 


Math 


060 


10.4% 


11 




13 




15 


Math 


070 


6.6% 


7 




16 




17 


Math 


140 


5.6% 


6 




18 




23 


Math 


102 


28.3% 


30 




24 




40 


Math 


211 


49.1% 


52 
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Placement Accuracy . Tables 3 through 9 are placement 
classification tables that show the relationships between 
admission status (that would have resulted from use of the 
recommended placement rules) and actual student performance 
observed in the course. Correct classification are 
represented in the upper right and lower left cells of the 
table, the sum of these cells represents the total of 
correct placement. The percentage of correct 
classifications is shown as a footnote to each table. In 
addition , the footnotes show the net gain in correct 
placements that would result from use of the recommended cut 
score and the percentage of students eligible (selection 
ratio) under this placement rule. 
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Table 3. 
Percentage of Students by 
Math 060 Success and Admission Status 

MDTP1 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 
Database: Math 01 F91 - F93 (Filter: NONE) 
Course: MATH 060 

Test: MATH 01: ALGEBRA READINESS 
Subgroup: TOTAL (N s 134) 



NOT ELIG ELIGIBLE 
< 30 >= 30 

















Correct Placements : 


59 


OX 








9 




43 


! 52 


Base Rate(non-succ): 


61 


2X 


SUCCESS 




6 


. 7X 




32. IX 


! 38. 8% 


Gain In Accuracy: 


-2 


2X 


A f B i C i C R 


r 


17 


.3% 


r 


82. 7X 


|r 100. OX 










c 


20 


.OX 


c 


48. 3% 
















36 








Selection Ratio: 


66 


4X 










46 


! 82 


Diff in Succ Ratio: 


28 


3% 


NOT SUCC 




26 


.9X 




34. 3% 


! 61. 2% 








D, F.NC.W 


r 


43 


.9% 


r 


56. IX 


!r 100. OX 










c 


80 


.OX 


c 


51 .7X 











45 89 134 

33. 6X 66. 4X 100. OX 

c 100. OX c 100. OX 



Table 4. 
Percentage of Students by 
Math 060 Success and Admission Status 

MDTP2 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 
Database: Math 02 F91 - F93 (Filter: NONE) 
Course: MATH 060 

Test: MATH 02: ELEMENTARY ALGEBRA C 
Subgroup: TOTAL (N = 226) 



NOT ELIG ELIGIBLE 





< 18 


>= 18 






21 




95 J 


SUCCESS 




9.3X 




42. OX } 


A.B.C.CR 


r 


18. IX 


r 


81. 9X Jr 




c 


26. 9X 


c 


64. 2X ' 






57 




53 i 


NOT SUCC 




25. i% 




23. 5X { 


D , F , NC , W 


r 


51 .8X 


r 


4 8. 2X |r 




c 


73. IX 


c 


35. 8X ! 



116 
51.3* 
100. OX 



110 
48. 7X 
100. OX 



78 148 226 

34. 5X 65. 5X 100. OX 

c 100. OX c 100.0% 



Correct Placements: 67. 3X 
Base Rate (success*: 51. 3X 
Gain in Accuracy: 15. 9X 



Selection Ratio: 
Diff in Succ Ratio: 



65. 5X 
3 7.3X 
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Table 5. 
Percentage of Students by 
Math 070 Success and Admission Status 

MDTP2 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 
Database: Math 02 F91 - F93 (Filter: NONE) 
Course: MATH 070 

Test: MATH 02: ELEMENTARY ALGEBRA C 
Subgroup: TOTAL (N = 115) 



NOT ELIG ELIGIBLE 
< 34 >* 34 















Correct Placements: 


60 


.0% 






28 




24 


52 


Base Rate ( non-aucc ) : 


54 


.8% 


SUCCESS 




24 . 3% 




20.9% 


4 5.2% 


Gain in Accuracy: 


5 


.2% 


A.B.C.CR 


r 


53.8% 


r 


46.2% 


r 100.0% 










c 


38.4% 


c 


57. 1% 






















Selection Ratio: 


36 


. 5% 






45 




18 


63 


Diff in Succ Ratio: 


18 


.8% 


NOT SUCC 




39. 1% 




15.7% 


54.8% 








D , F , NC , W 


r 


71.4% 


r 


28.6% 


r 100.0% 










c 


61 .6% 


c 


42.9% 














73 




42 


115 












63.5% 




36.5% 


100.0% 









c 100.0% c 100.0% 

Table 6. 
Percentage of Students by 
Math 070 Success and Admission Status 

MDTP3 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 
Database: Math 03 F91 - F93 (Filter: NONE) 
Course: MATH 070 

Test: MATH 03: INTERMEDIATE ALGEBRA C 
Subgroup: TOTAL (N = 171) 



NOT ELIG ELIGIBLE 
< 21 >= 21 















Correct Placements: 


6 3 


7% 






42 




39 


81 


Base Rate( non-succ ) : 


52 


6% 


SUCCESS 




24.6% 




22.8% 


47.4% 


Gain in Accuracy: 


11 


1% 


A.B.CCR 


r 


51.9% 


r 


48.1% 


r 100.0% 










c 


37 . 5% 


c 


66.1% 






















Selection Ratio : 


34 


5% 






70 




20 


90 


Diff in Succ Ratio: 


28 


.6% 


NOT SUCC 




40.9% 




11.7% 


52.6% 








D.F.NC.W 


r 


77.8% 


r 


22.2% 


r 100.0% 










c 


62 . 5% 


c 


33.9% 











112 59 171 

65. 5X 34 . 5X 100. OX 

c 100. OX c 100. OX 
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Table 7. 
Percentage of Students by 
Math 102 Success and Admission Status 

MDTP3 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 
Database: Math 03 F91 - F93 (Filter: NONE) 
Course: MATH 102 

Test: MATH 03: INTERMEDIATE ALGEBRA C 
Subgroup; TOTAL <N • 70) 



NOT ELIG ELIGIBLE 
< 29 >« 29 















Correct Placements: 


57 


1% 






21 




13 


34 


Base Rate(non-succ): 


51 


4% 


SUCCESS 




30. or. 




18.6% 


4 8.6% 


Gain in Accuracy: 


5 


7% 


A.B.CCR 


r 


61 .8% 


r 


38.2% 


r 100.0% 










c 


43. 8% 


c 


59.1% 






















Selection Ratio: 


31 


4% 






27 




9 


36 


Diff in Succ Ratio: 


15 


3% 


NOT SUCC 




38. 6% 




12.9% 


51.4% 








D , F , NC , W 


r 


75.0% 


r 


25.0% 


r 100.0% 










c 


56. 3r. 


c 


40.9% 














48 




22 


70 












68. 6% 




31.4% 


100.0% 









c 100.0% c 100.0% 



Table 8. 
Percentage of Students by 
Math 102 Success and Admission Status 

MDTP4 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 

Database: Math 04 F91 - F93 (Filter: NONE) 

Course: MATH 102 

Test: MATH 04: PRECALCULUS C 

Subgroup: TOTAL (N * 23) 



NOT ELIG ELIGIBLE 
< 19 >= 19 



















Correct Placements: 


69 


6% 








3 






13 


16 


Base Rate (success): 


69 


6% 


SUCCESS 




13 


0% 




56 


.5% 


69.6% 


Gain in Accuracy: 


0 


0% 


A ( B,C,CR 


r 


18 


8% 


r 


81 


.3% 


r 100.0% 










c 


50 


0% 


c 


76 


.5% 


























Selection Ratio: 


73 


9% 








3 






4 


7 


Diff in Succ Ratio: 


26 


5% 


NOT SUCC 




13 


0% 




17 


.4% 


30.4% 








D,F,NC,W 


r 


42 


9% 


r 


57 


.1% 


r 100.0% 










c 


50 


0% 


c 


23 


.5% 
















6 






17 


23 












26 


1% 




73 


.9% 


100.0% 









c 100.0% c 100.0% 



BEST COPY AVAILABLE 
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Table 9. 
Percentage of Students by 
Math 211 Success and Admission Status 

MDTP4 

CLASSIFICATION TABLES FOR OPTIMAL CUT SCORES 

Database: Hath 04 F91 - F93 (Filter: NONE) 

Course: MATH 211 

Test: MATH 04: PRECALCULUS C 

Subgroup: TOTAL {N = 40) 



NOT ELIG ELIGIBLE 
< 25 >= 25 













Correct Placements: 


82 


5X 






0 


26 


26 


Base Rate ( success ) 


65 


OX 


SUCCESS 


0 


OX 


65. OX 


65. OX 


Gain in Accuracy: 


17 


5X 


A , B, C , CR 


r 0 


Ox 


V 10 0. OX 


r 100 . OX . 










c 0 


OX 


c 78. 8X 




















Selection Ratio: 


82 


5X 






7 


7 


14 


Diff in Succ Ratio: 


78 


8X 


NOT SUCC 


17 


5% 


17. 5% 


35. OX 








D,F,NC,W 


r 50 


or. 


r 50. OX 


r 100. OX 










c 100 


0% 


c 21. 2X 














7 


33 


40 










17 


.5* 


82. 5X 


1 00 . OX 










c 100 


OX 


c 100. OX 
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The following line graphs provide a visual depiction of 
three of the major factors considered when selecting the 
appropriate cut scores: (1) percentage of correct 
placements; (2) percentage of false positives - or those who 
were eligible but who received an unsuccessful final grade; 
and (3) percentage of false negatives - or those who were 
ineligible but received a successful final grade. One 
common error in selecting cut scores is to only look at the 
percentage of correct placements. By doing so, it 
encourages cut scores that are too high. Figures 8 through 
14 show the recommended cut scores. 
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Figure 8. 

Math 060 Recommended Cut Score of 30 

MDTP1 
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Figure 9. 

Math 060 Recommended Cut Score of, 18 

MDTP2 
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Figure 10. 
Math 070 Recommended Cut Score of 34 
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Figure 11* 
Math 070 Recommended Cut Score of 21 

MDTP3 
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Figure 12. 
Math 102 Recoitjnended Cut Score of 29 

MDTP3 
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Figure 13. 
Math 102 Recommended Cut Score of 19 

MDTP4 
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Figure 14. 
Math 211 Recommended Cut Score of 25 

MDTP4 
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Table 10 provides supplemental information in support of the 
relationship between test scores and grades. Specifically, 
the mean test scores for key criterion (grade) defined 
groups — for students earning an A or B grade, a C grade, a 
grade of D or F, and those receiving Ws — are presented. 
The mean test scores for the highly successful students 
(those witn A or B grades) is clearly greater than the mean 
values for students who earned C grades, who in turn tended 
to have higher scores than those with D or F grades. While 
the pattern is not a perfect linear relationship, these data 
are consistent with the overall linear relationships between 
test scores and course grades, and also support the 
reasonableness of the recommended cut score levels. 
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Table 10. 

Mean Score for Key Grade-Based Criterion Groups, 

By Course 



Final Grade Earned 
Test/Course A/B C D/F W 



MDTP1 

Math 025 
Math 057 
Math 060 



19.85 
27.71 
35.86 



17.43 
32.76 



16.83 
25.91 
33.15 



16.91 
25. 33 
29.96 



MDTP2 

Math 025 
Math 060 
Math 070 



* 

22.84 
33.13 



* 

21.79 
30.50 



9.50 
19.38 
32.00 



18.23 
29.29 



MDTP3 

Math 060 
Math 070 
Math 102 
Math 103 
Math 140 
Math 240 



13.12 
20.31 
27.74 
28.60 
23.43 
31.40 



11.91 
19.62 
27.00 
31.38 
23.00 



11.07 
18.17 
27.73 
25.00 
22.67 
* 



13 .00 
16.75 
25.95 
25.00 
27.60 



MDTP4 

Math 060 
Math 070 
Math 102 
Math 103 
Math 140 
Math 211 
Math 240 



16.40 
23 . 33 
25.67 
23.67 
30.65 
* 



18.71 

17.20 
* 

28.00 
* 



* 
* 

17.83 

* 
* 
* 



* 

24 .40 



N is 5 or less, 
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Since the study samples represent only a portion of the 
students who actually took each of the MDTP tests during 
these five semesters, we questioned what proportion of the 
total groups would have fallen into each of the recommended 
cut score ranges. Table 11 provides the breakdowns. 
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Table 11. 

Recommended MDTP Cut Scores by Course 
All Tested Students, Fall 1991 to Fall 1993 



Test 


Score 


Range 


Recommended 
Placement 


% Of 
Total Tested 


N 


MDTP1 


0 


- 


29 


Math 


025 


71. 6% 


1563 




30 


- 


50 


Math 


060 


28 . 4% 


619 


MDTP2 


0 


- 


17 


Math 


025 


42.4% 


492 




18 


- 


33 


Math 


060 


48.5% 


563 




34 


- 


50 


Math 


070 


9 . 1% 


106 


MDTP3 


0 




9 


Take 


MDTP2 


10. 2% 


87 




10 




20 


Math 


060 


48.9% 


417 




21 




28 


Math 


070 


28.7% 


245 




29 




45 


Math 


102 


12.2% 


104 


MDTP 4 


0 




9 


Take 


MDTP3 


5.8% 


13 




10 




18 


Math 


070 


28.9% 


65 




19 




24 


Math 


102 


24.0% 


54 




25 




40 


Math 


211 


41 . 3% 


93 
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A smaller percentage of the total group of the MDTP1 
and MDTP2 test takers fell into the Math 060 score ranges 
than was true of the study sample (MDTP1: 28.4% versus 
46.8% and MDTP2: 48.5% versus 57.0%). 

DISCUSSION 

This study provides clear evidence supporting the 
reasonableness of the recommended cut scores for each of the 
four math placement tests. The cut scores should be 
monitored and adjusted as needed. 

A thorough review of these findings by all involved 
faculty and Student Services personnel is encouraged. 

RECOMMENDATIONS 
1. Eliminate the use of sub-scores in placement . The 
college currently uses a cut score system for the MDTP tests 
which is an adaptation of the system initially used by Santa 
Barbara City College (SBCC). The system makes use of both 
an overall score and scores on sub-sets of questions within 
a test to make a judgement about course placement. 

In discussing the development of the scoring system 
with Robert Elmore at SBCC, he indicated that their college 
had run scatter diagrams to develop the best overall cut 
scores back in 1985-86, but had never carried out any 
validation research on the sub-scores. SBCC no longer uses 
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the sub-scores in their math placement process* 

The sub-scores offer additional information about the 
specific areas of strength or weakness. The sub-scores 
essentially become mini-placement tests and as such would 
require extensive validation study. 

2. Provide sub-score feedback to students . Each test 
contains several questions designed to test the math 
knowledge or ability of a student in a given area. The 
sub-tests are then added together to form an overall 
placement test score. 

The college should provide each student with a report 
on the diagnostic sub-scores he/she receives on the MDTP 
test. The CAPP system provides the option of printing a 
personal letter to a student which provides diagnostic 
feedback as to the strengths and deficiencies found (ASAP 
Manual, page 75). The letter includes the students overall 
math score and provides feedback on the areas in which the 
student did well and the areas in which he/she needs 
review. The letter also provides comparative information 
for each of the sub-scales, indicating the total possible 
points available and the level that is considered a mastery 
level. For example, a student who took the Pre-calculus 
test may do well in rational expressions and linear 
equations but may need review in exponents and radicals, 
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polynomials, functions, trigonometry, logs and exponential 
functions and word problems. The letter also provides space 
for the college to indicate where assistance is available to 
the student, should they not be able to enroll in a 
recommended math course that semester. 

3. Provide sub-score information to faculty . The 
matriculation program could provide valuable assistance to 
individual math faculty if they were able to provide them 
with copies of Diagnostic Sub-scores letters for each of the 
students enrolled in 'cheir class. The test results provide 
a faculty member with a clear indication of the areas in 
which the class has strengths and the areas in which it 
needs assistance. 

CUTMDTP. FW4 
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